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We  present  a  sequential  approach  to  detect  static  targets  with  imperfect  sensors,  which  range  from 
tower-mounted  cameras  to  satellites.  The  scenario  is  operationally  relevant  to  many  military,  homeland 
security,  search  and  rescue,  environmental  engineering,  counter-narcotics,  and  law  enforcement  applica¬ 
tions.  The  idea  is  to  stop  the  search  as  soon  as  there  is  enough  probabilistic  evidence  about  the  targets' 
locations,  given  an  operator-prescribed  error  tolerance,  knowledge  of  the  sensors’  parameters,  and  a 
sequence  of  detection  signals  from  the  sensors.  By  stopping  the  search  as  soon  as  possible,  we  promote 
efficiency  by  freeing  up  sensors  and  operators  to  perform  other  tasks.  The  model  we  develop  has  the 
added  benefits  of  decreasing  operator  workload  and  providing  negative  information  as  a  search 
progresses. 
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1.  Introduction 

Today’s  operational  planners  and  sensor  operators  face  numer¬ 
ous  challenges  inherent  to  the  complex  environments  that  shape 
their  space  of  operation.  These  challenges  are  further  magnified 
by  scarce  resources,  imperfect  information,  and  operator  task 
overload. 

The  time  critical  nature  of  the  command  decisions  that  serve  as 
milestones  throughout  the  Find,  Fix,  Track,  Target,  and  Engage 
(F2T2E)  process  further  exacerbate  the  situation.  Defense  planners 
must  strive  to  develop  and  incorporate  new,  efficient  procedures 
to  allocate  scarce  resources  in  many  different  complex  environ¬ 
ments.  Any  efficiency  gained  within  the  F2T2E  chain,  however 
small,  may  have  a  compound  effect  over  time  on  overall  opera¬ 
tional  readiness  because  this  will  free  up  assets  to  perform  other 
time-sensitive,  critical  sensing  actions,  as  well  as  decrease  operator 
workload  and  capitalize  upon  negative  information.  Such  negative 
information  could  be  utilized  to  find  where  targets  are  not  located, 
and  may  help  determine  areas  to  set  up  certain  operations  or  paths 
through  the  environment  that  are  free  of  hostile  forces. 

In  this  article  we  consider  a  scenario  with  multiple  fixed-sen¬ 
sors  and  multiple  static  targets  in  discrete-time  and  discrete-space. 
The  sensors  may  range  from  tower-mounted  cameras,  to 
Unmanned  Aircraft  Systems  (UASs),  to  satellites,  and  the  targets 
under  consideration  do  not  react  to  any  sensing  action.  The 
scenario  is  operationally  relevant  to  many  military,  homeland 
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security,  search  and  rescue  (SAR),  environmental  engineering, 
counter-narcotics,  and  law  enforcement  applications.  UASs  have 
been  used  in  Iraq  and  Afghanistan  to  search  for  Improvised 
Explosive  Devices  (IEDs),  insurgent  safe  houses,  suspected  weap¬ 
ons  caches,  and  mortar  points  of  origin  [11,12].  Other  relevant 
applications  include  searching  for  downed  aircraft  or  life  rafts, 
detecting  illegal  drug  harvesting  and  processing  operations,  patrol¬ 
ling  border  infiltration  points,  and  tracking  flora  and  fauna  counts 
in  biological  environments. 

We  formulate  a  model  to  locate  static  targets  of  interest  (TOIs) 
hidden  within  an  area  of  interest  (AOI).  As  in  reality,  our  model  al¬ 
lows  the  analyst  to  contend  with  the  fact  that  the  search  sensors 
are  imperfect;  i.e.,  the  sensors  may  declare  fewer  or  more  targets 
than  are  actually  present  on  a  particular  search  attempt.  The  idea 
is  to  stop  the  search  as  soon  as  there  is  enough  probabilistic  evi¬ 
dence  about  the  TOIs’  locations,  given  an  operator-prescribed  error 
tolerance,  knowledge  of  the  sensors’  parameters,  and  a  sequence  of 
detection  signals  from  the  sensors. 

The  AOI  for  the  scenario  is  comprised  of  a  grid  of  discrete,  non¬ 
overlapping  area-cells  (ACs).  The  area-cells  might  be  defined  by 
geo-political  borders,  terrain  features,  or  some  arbitrary  grid  sys¬ 
tem  of  tactical  significance  to  the  operator,  and  need  not  be  uni¬ 
form  in  size  nor  shape.  Each  cell  is  characterized  by  the  number 
of  sensors  (known),  the  sensors’  operational  parameters  (known), 
and  the  number  of  targets  (unknown). 

The  sensor  parameters  are  the  conditional  probabilities  of 
returning  each  possible  detection  signal  given  each  possible  num¬ 
ber  of  actual  TOIs  in  that  area-cell.  More  specifically,  when  the 
operator  makes  an  investigation  into  an  area-cell,  the  sensor  re¬ 
turns  a  detection  signal  corresponding  to  the  number  of  TOIs  seen 
by  the  sensor  with  some  probability  that  depends  on  the  actual 


0377-2217/$  -  see  front  matter  Published  by  Elsevier  B.V. 
doi:10.1016/j.ejor.201 1.05.045 


Please  cite  this  article  in  press  as:  ICE.  Wilson  et  al.,  A  sequential  perspective  on  searching  for  static  targets,  European  Journal  of  Operational  Research 
(2011),  doi:  10.1 01 6/j.ejor.201 1.05.045 


Report  Documentation  Page 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 


1.  REPORT  DATE 

2011 


2.  REPORT  TYPE 


4.  TITLE  AND  SUBTITLE 

A  Sequential  Perspective  On  Searching  For  Static  Targets 


6.  AUTHOR(S) 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Naval  Postgraduate  School, Operations  Research 
Department, Monterey, CA, 93943 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


3.  DATES  COVERED 

00-00-2011  to  00-00-2011 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

5d.  PROIECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

European  Journal  of  Operational  Research,  preprint,  2011 

14.  ABSTRACT 

We  present  a  sequential  approach  to  detect  static  targets  with  imperfect  sensors,  which  range  from 
tower-mounted  cameras  to  satellites.  The  scenario  is  operationally  relevant  to  many  military,  homeland 
security,  search  and  rescue,  environmental  engineering,  counter-narcotics,  and  law  enforcement 
applications.  The  idea  is  to  stop  the  search  as  soon  as  there  is  enough  probabilistic  evidence  about  the 
targets?locations,  given  an  operator-prescribed  error  tolerance,  knowledge  of  the  sensors?  parameters,  and 
a  sequence  of  detection  signals  from  the  sensors.  By  stopping  the  search  as  soon  as  possible,  we  promote 
efficiency  by  freeing  up  sensors  and  operators  to  perform  other  tasks.  The  model  we  develop  has  the  added 
benefits  of  decreasing  operator  workload  and  providing  negative  information  as  a  search  progresses. 

15.  SUBIECT  TERMS 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 

18.  NUMBER 

19a.  NAME  OF 

ABSTRACT 

OF  PAGES 

RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Same  as 
Report  (SAR) 

10 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


ARTICLE  IN  PRESS 


2  K.E.  Wilson  et  al./ European  Journal  of  Operational  Research  xxx  (201 1 )  xxx-xxx 


(i.e.,  the  ground  truth,  which  is  unknown)  number  of  TOIs  in  that 
area-cell. 

To  efficiently  determine  the  targets’  locations  subject  to  the 
operator-prescribed  error  tolerance,  we  develop  a  sequential  elim¬ 
inating  procedure  [14).  A  sequential  eliminating  procedure  at¬ 
tempts  to  isolate,  from  among  several  candidate  configurations, 
one  particular  desired  configuration— the  objective.  During  a  par¬ 
ticular  stage  of  a  sequential  eliminating  procedure,  all  candidate 
configurations  are  examined  and  ranked  in  order  of  their  likelihood 
of  producing  the  sequence  of  observed  signals  up  to  that  stage. 

Any  configuration  whose  likelihood,  when  compared  with  the 
configuration  of  maximum  likelihood,  exceeds  a  particular  thresh¬ 
old  (which  depends  on  the  user’s  error  tolerance)  is  permanently 
eliminated  from  the  set  of  candidates.  If  no  configuration  exceeds 
the  threshold  during  a  particular  stage,  then  all  those  configura¬ 
tions  remain  in  the  set  of  candidates.  The  procedure  advances  to 
the  next  stage,  using  the  updated  candidate  set.  The  process  con¬ 
tinues  until  only  one  configuration  remains  in  the  candidate  set, 
and  that  configuration  is  declared  the  winner.  In  our  case,  the  con¬ 
figurations  are  the  ways  the  TOIs  can  be  located  in  the  area-cells  of 
the  AOI,  and  the  winner  is  the  determined  configuration.  We  desig¬ 
nate  the  actual  location  of  the  targets  in  the  area-cells  to  be  the 
ground  truth  configuration  (GTC). 

Search  theory  [1]  traces  its  roots  to  the  pioneering  work  of 
Koopman  [5].  For  the  search  scenario  we  focus  on,  the  objective 
is  to  locate  targets  within  a  finite  number  of  cells  [1],  In  this  case, 
searcher  success  is  achieved  by  either  detecting  the  targets,  or,  if 
the  targets  are  not  detected,  by  correctly  guessing  the  cells  con¬ 
taining  the  targets.  Tognetti  [17]  and  Kadane  [4]  treat  the  scenario 
of  whereabouts  search  against  a  stationary  target.  Washburn  [19] 
is  a  classical  reference  in  search  theory. 

Siegmund’s  1985  book  [14]  is  the  classic  reference  in  sequential 
analysis,  and  deals  primarily  with  sequential  hypothesis  testing 
and  related  problems  of  estimation.  In  many  of  these  cases,  a 
fixed-sample  solution  exists,  but  one  can  employ  sequential  meth¬ 
ods  to  achieve  greater  efficiency  in  the  solution.  Siegmund  presents 
a  sequential  test  with  the  same  power  as  a  fixed-sample  test  and 
requires  fewer  observations  [14].  Therefore,  the  sequential  test 
has  a  reasonable  claim  to  be  regarded  as  more  efficient  [14J. 

While  the  work  to  date  in  selection  using  sequential  eliminating 
procedures  [9,10,20]  has  focused  on  isolating  the  best  system  - 
usually  the  one  with  a  maximum  unknown  parameter  value  - 
our  goal  is  instead  to  isolate  one  determined  configuration.  The  de¬ 
sire  is  for  the  determined  configuration  to  be  the  ground  truth  con¬ 
figuration.  That  is  it  correctly  specifies  the  number  of  TOIs  in  each 
area-cell.  We  show  that  our  sequential  model  provides  determined 
configurations  efficiently,  while  guaranteeing  to  meet  the  user- 
prescribed  error  tolerance. 

Compared  to  existing  sensor  employment  models  (e.g. 
[6,13,7,15]),  our  approach  does  not  consider  moving  targets,  does 
not  dynamically  allocate  the  sensors  (i.e.,  no  decision  is  taken  as 
to  where  the  sensors  look  in  each  stage),  and  does  not  find  optimal 
search  paths  (with  or  without  restrictions  on  searching  area-cells 
within  a  vicinity  of  the  last  searched  area-cell).  Delving  into  the 
last  point,  most  recent  models  (e.g.  [6,13,7])  employ  optimization 
techniques  (deterministic  or  stochastic)  that  yield  search  paths 
that  are  optimal  in  a  certain  sense  (often,  but  not  always,  maximiz¬ 
ing  the  expected  number  of  detected  targets)  for  a  prescribed  num¬ 
ber  of  time  periods  or  search  effort.  However,  these  models  scale 
poorly  and  become  intractable  even  for  a  relatively  small  number 
of  TOIs,  area-cells,  and  time  periods  under  consideration.  This  oc¬ 
curs  because  the  computational  cost  grows  at  least  exponentially 
in  the  number  of  variables  ([18]),  which  generally  is  #TOIs  x 
#ACs  x  #time  periods.  Some  heuristics  (e.g.  [16])  have  been  pro¬ 
posed  to  overcome  this  difficulty,  but  their  performance  cannot 
be  theoretically  guaranteed.  While  our  approach  does  provide 


some  benefits  over  existing  methods,  it  too  has  limitations  with  re¬ 
gard  to  the  the  size  of  problems  it  can  be  applied  to.  The  algorithm 
can  only  be  used  in  situations  where  there  are  a  handful  of  TOIs 
(e.g.  less  than  5)  because  it  must  consider  all  possible  configura¬ 
tions  of  TOIs  in  the  area  cells.  There  are  many  applications  where 
this  is  the  case  (e.g.  SAR  scenarios,  searching  for  an  insurgent  safe 
house)  and  our  algorithm  would  provide  an  appropriate  and  effec¬ 
tive  approach. 

Once  again,  this  article  presents  a  sequential  perspective  on 
imperfect  sensor  employment  applicable  to  static  targets  that  is 
easy  to  implement,  is  computationally  tractable  when  there  are  a 
small  number  of  TOIs  for  a  larger  class  of  problems  than  the  opti¬ 
mization  approaches  currently  being  employed,  and  stops  when 
the  user  prescribed  probabilistic  guarantees  are  met  (and  thus 
the  number  of  time  periods  or  search  effort  is  an  output  of  the 
model). 

The  article  is  organized  as  follows.  Section  2  introduces  the 
notation  and  key  definitions.  In  Section  3  we  present  the  sequential 
approach.  Section  4  shows  numerical  illustrations  of  the  model, 
and  Section  5  closes  the  paper  with  the  main  conclusions. 

2.  Notation 

In  this  section  we  introduce  the  notation  that  will  be  employed 
throughout  this  article. 

A:  Number  of  area-cells. 

M:  Number  of  targets  of  interest. 

m,-:  Number  of  targets  of  interest  in  area-cell  i.  This  value  is 
unknown,  and  is  what  the  analyst  wishes  to  determine  for 
i  =  l,...,A. 

m:  The  true  configuration,  m  =  (_mt,m2,. . .  ,mA ). 

C:  The  set  of  feasible  target  configurations,  formed  by  the  ele¬ 
ments  t  =  (U,...,tA)  non-negative  and  integer  such  that 

Et  = 

K:  The  number  of  feasible  target  configurations  I<  =  |C|  = 
M  +  A-  1\ 

M  )■ 

Pi(d\tj):  Conditional  probability  that  the  sensor  in  area-cell  i 
returns  a  signal  “d  targets"  given  that  t,  TOIs  are  present  there. 
S,:  The  sensor  present  in  AQ  is  completely  characterized  by  the 
(M  +  1 )  x  (iW  +  1 )  matrix  Sf.  The  value  in  the  tth  row  and  dth  col¬ 
umn  of  Si  is  the  probability  p,(d|t).  The  matrix  S,  is  stochastic,  so 
the  elements  of  each  row  constitute  a  probability  mass 
function. 

X,,i,X,,2, . . Sequence  of  signals  returned  by  the  sensor  in  ACf, 
independently  and  identically  distributed  (I1D)  with  probability 
mass  function  p>(-jmf). 

£  (x,  i, . . .  ,xln  ;  t,-):  For  n  I1D  signals  from  the  sensor  in  AC„  Xf  l  =  x- 
i,i  =  x,  i,Xi2  =  x,2, . . . ,X,n  =  xI>n,  the  likelihood  of  having  tf  targets 
in  area-cell  i  is 

n 

^(Xi, X, ti)  =  nPi(xult')- 

J=1 

3.  Model 

We  describe  the  model  in  more  detail  in  section  3.1.  Our  objec¬ 
tive  is  to  analyze  the  scenario  of  multiple  targets  in  an  area  of 
interest  with  many  cells.  This  produces  many  potential  configura¬ 
tions.  This  case  is  difficult  to  analyze  because  knowledge  about  the 
presence/absence  of  targets  in  an  area-cell  yields  insight  about  the 
presence/absence  of  targets  in  other  area-cells;  i.e.,  the  number  of 
targets  in  each  area-cell  is  not  independent.  To  gain  insight  into  the 
problem,  in  Section  3.2  we  examine  the  scenario  where  there  are 
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only  two  possible  configurations.  In  Section  3.3  we  present  the 
sequential  eliminating  algorithm,  and  in  Section  3.4  we  prove  that 
this  algorithm  will  finish  in  a  finite  number  of  steps  with  probabil¬ 
ity  1. 

3.1.  Background 

We  analyze  a  situation  consisting  of  M  TOIs  located  in  A  area- 
cells,  and  area-cell  i  contains  m,  targets.  The  number  of  targets  in 
each  cell  can  be  any  value  between  0  and  M  as  long  as 
i  =  1W;  other  constraint  scenarios,  such  as  imi  <  M,  can 
be  treated  by  enlarging  the  number  of  possible  configurations. 
We  denote  the  set  of  possible  configurations  of  the  M  targets  in 

the  A  area-cells  as  C  and  therefore  \C\  =  +  ^  ~  ^  (see  page 

38  of  Feller  [3]  for  a  derivation).  There  is  one  sensor  in  each  cell 
that  provides  an  imperfect  signal  to  the  sensor  operator  about 
the  number  of  targets  in  each  cell.  This  model  can  allow  for  multi¬ 
ple  sensors  in  each  cell,  but  for  notational  simplicity  we  only  ana¬ 
lyze  one  sensor  in  each  cell.  The  probability  the  sensor  in  area-cell  i 
produces  a  signal  that  the  area-cell  contains  d  targets  is  pj(d[m>). 
We  construct  a  sensor  matrix  S,  in  which  the  (t,d)  element  is  given 
by  p,(d|t).  Each  row  ofS;,  p,(-|t),  is  a  probability  mass  function  (pmf). 
Thus  the  matrix  S,  fully  characterizes  the  sensor  in  AC,.  We  assume 
that  for  all  area-cells  1  sj  i  ^  A  the  values  p,(d|t)  are  known  and  that 
any  two  rows  of  the  sensor  matrix  S,  have  at  least  one  different 
element. 

Since  the  sensors  are  imperfect,  it  is  desirable  to  frame  the  esti¬ 
mate  with  some  level  of  confidence.  To  accomplish  this,  the  oper¬ 
ator  prescribes  an  error  tolerance.  For  example,  an  error  tolerance 
of  5  percent  would  indicate  that  the  operator  is  willing  to  accept 
that  the  model  returns  an  acceptable  configuration  at  least  95  per¬ 
cent  of  the  time.  In  this  paper,  the  only  acceptable  configuration  is 
the  ground  truth  configuration. 

The  objective  for  the  sensor  operator  (and  the  one  considered  in 
this  article)  is  to  find  the  ground  truth  configuration  subject  to 
bounds  on  the  probability  of  returning  the  wrong  configuration. 
Naturally,  the  larger  the  error  tolerance  the  operator  is  willing  to 
accept,  the  more  quickly  the  operator  can  expect  the  procedure 
to  complete.  Conversely,  a  small  tolerance  for  error  could  mean 
many  more  search  attempts  are  expended  before  the  procedure 
terminates. 

A  key  component  of  the  sequential  eliminating  procedure  is  cal¬ 
culating  the  likelihood  ratios  between  various  configurations.  The 
sequence  of  signals  returned  by  the  sensor  in  AQ  is  an  1ID  sequence 
of  random  variables  Xu,Xjp2,...  drawn  from  the  probability  mass 
function  pi(-|nij).  If  the  realization  of  these  signals  is 
Xi  i  =  Xjj , Xl  2  =  x!p2, . . .,  for  ACs  i  =  1, . . .  ,A,  then  the  joint  likelihood 
function  after  n  looks  in  each  area-cell  is  given  by 

A  n 

£(Xij,  Is S  i  <  A  1  <  j  <  n;  f,, . . . ,  tA)  =  I]  (1) 

i=i  j=i 

where  corresponds  to  an  arbitrary  configuration.  When 

the  rows  of  each  sensor  matrix  S,  are  different,  the  likelihood  func¬ 
tion  is  maximized,  as  n  ->  oo,  by  Q  =  mltt2  =  m2,. .  .,tA  =  mA ;  see  Leh¬ 
mann  [8]  for  the  general  theory.  For  two  arbitrary  configurations  t(0) 
and  A1,  their  likelihood  ratio  is 

*(xy,l  1  <js:n;  tf\ . . . ,  t<0)) 

1  <  i  <  A.  1  <  j  <  n;  t(f, . . . ,  t^y 


3.2.  Comparing  two  configurations 

In  this  section  we  analyze  the  special  case  where  there  are  are 
two  possible  configurations,  labeled  Configuration  0  and  Configu¬ 
ration  1,  only  one  of  which  is  the  GTC.  Configuration  0  has 
t(°)  =  jf(°)  fTOj  jois  located  among  the  area-cells,  and 

Configuration  1  t(1)  =  in  the  area-cells.  If  the 

ground  truth  configuration  is  Configuration  0,  then  the  signals  pro¬ 
duced  by  the  sensor  in  area-cell  i  will  be  distributed  according  to 
the  pmfp,(-|f|0)).  If  the  ground  truth  configuration  is  Configuration 
1,  then  the  signals  will  be  distributed  according  to  the  pmfp,(-|t!1)). 
To  guarantee  that  the  algorithm  terminates  in  finite  time,  we  as¬ 
sume  that  at  least  one  of  the  sensors’  matrices  has  its  t[0)  and  tA 
rows  different;  otherwise,  the  sequence  of  signals  produced  by 
the  sensors  would  be  probabilistically  identical  under  both 
configurations. 

We  assume  that  all  area-cells  receive  the  same  number  of  looks 
from  their  corresponding  sensor,  and  the  goal  is  to  terminate  the 
inspection  when  there  is  enough  evidence  that  the  error  bounds 
are  met,  i.e.,  we  are  confident  to  within  our  error  tolerances  of  say¬ 
ing  that  the  TOIs  are  located  in  the  area-cells  according  to  one  of 
the  two  configurations.  The  two  possible  errors  are:  (i)  a  is  the 
probability  that  Configuration  1  is  the  determined  configuration 
when  the  GTC  is  Configuration  0,  and  (ii)  p  is  the  probability  that 
Configuration  1  is  the  determined  configuration  when  the  GTC  is 
Configuration  0.  This  leads  to  the  hypotheses 

Ho  ■  Configuration  0  is  the  GTC,  and  H\ 

:  Configuration  1  is  the  GTC. 

The  likelihood  that  the  signals  Xy  =  Xy  are  produced  by  Configura¬ 
tion  0  is  (cf„  Eq.  (1)) 


and  the  likelihood  the  signals  are  produced  by  Configuration  1  is 

4°  =  1  ^  i  <  A  1  <  j  <  n;  t{'\ ...,  4°) . 

The  likelihood  ratio 


with  probability  1  as  n  ->  oo  if  the  GTC  is  Configuration  0  and 
£„  ->  ex)  if  the  GTC  is  Configuration  1 ;  see  Lehmann  [8],  This  suggests 
that  a  judicious  policy  is  to  stop  sampling  when  the  likelihood  ratio 
crosses  an  upper  threshold  and  declare  Configuration  1  the  deter¬ 
mined  configuration  or  when  the  likelihood  ratio  crosses  a  lower 
threshold  and  declare  Configuration  0  the  determined  configura¬ 
tion.  This  approach  may  lead  to  an  incorrect  determination,  but 
its  probability  can  be  prescribed  ab  initio  by  the  end-user. 

Define  the  stopping  time 

N=inf{n  >  1  :£„t(A,B)} 

for  threshold  constants  A,  B  such  that  0  <  A  <  B  <  oo.  Then  we 

Reject  Ho  if  A  >  B.  and  Accept  Ho  if  A  <  A. 

For  0  ^  a,  p  ^  1  prescribed  by  the  end-user,  the  error  probabilities 
are 

Type  I  error  :  P(A  >  B\Ho)  =  a  and  Typell  error  :  P( A 
A\Hi)  =  p. 


which  is  always  a  number  in  (0,oo)  for  finite  n  if  p,{xylti)  >  0.  Taking  logarithms,  we  can  see  that  we  reject  Ho  if 
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Y  Y  (1°gPi(x.j|tP))  -  >  logB  (2) 

j=  1  '=1 

and  accept  Ho  if 

Y  Y  (loS  Pi  (xij  I  £i 1  ’ )  -  |ogpi(x,j|t|0)))  <  log  A.  (3) 

j= 1  '=1 

Siegmund  [14]  shows 

a  =  P(4  >  B\H0)  <B’(1  -P)  (4) 

and 

P  =  P(£n^A\Hi)  -a).  (5) 

Hence,  given  operator  defined  tolerances  a  and  /?,  by  setting 

B  =  - — and  A  =  — ,  (6) 

a.  1  -  a 

we  are  guaranteed  to  satisfy  the  error  probability  constraints.  More¬ 
over,  it  is  shown  in  Siegmund  that  if  Eqs.  (4)  and  (5)  hold  with 
equality,  then  this  approach  minimizes  the  expected  number  of 
looks  until  crossing  either  boundary  [14].  Although  (4)  and  (5)  gen¬ 
erally  do  not  hold  with  equality,  the  algorithm  is  guaranteed  to 
meet  the  error  criteria. 

Fig.  1  displays  two  possible  sample  paths  for  this  sequential 
eliminating  procedure.  The  two  parallel  dashed  lines  represent 
the  bounds  log  (A)  and  log  (B).  A  path  between  the  bounds  is  still 
undetermined;  thus  the  procedure  continues  until  the  path  exits 
via  one  of  the  bounds  (hence,  this  area  is  known  as  the  continuation 
region ).  A  path  exiting  the  bound  corresponding  to  log  (B)  results  in 
a  declaration  that  Configuration  1  is  the  determined  configuration, 
whereas  an  exit  via  the  log  (A)  bound  results  in  a  declaration  that 
Configuration  0  is  the  determined  configuration. 

3.3.  Sequential  eliminating  algorithm 

We  now  consider  the  general  scenario  of  an  AOI  consisting  of 
A  >  1  area-cells,  and  M  targets  located  within  the  AOI.  There  are 

I<  =  ^  M  +  ^  _  ^  ways  for  the  M  targets  to  be  placed  in  the  A  cells, 

leading  to  configurations  P\f2\. .  .,(**1  Assume,  without  loss  of 
generality,  that  the  GTC  is  Configuration  1. 

An  incorrect  determination  occurs  when  the  determined  config¬ 
uration  is  not  Configuration  1.  We  call  this  event  an  ICD  (for  incor¬ 


rect  determination ),  and  the  desire  is  for  P(ICD)  <  a,  for  some 
a  6  (0,1)  pre-specihed  by  the  operator. 

Let  Co  be  the  initial  set  of  candidate  configurations;  initially  all  K 
configurations  are  candidates  to  contain  the  target,  so  that  C0  =  C. 
Let  tn  be  the  largest  likelihood  at  stage  n.  We  drop  a  configuration 
from  consideration  when  there  is  sufficient  evidence  that  it  it  is  not 
the  correct  configuration,  i.e.,  when  we  are  confident  to  within  our 
error  tolerance  of  saying  that  the  targets  are  not  located  according 
to  that  particular  configuration.  This  suggests  eliminating  configu¬ 
ration  r  when  f*/4r>  5=  B,  where  B  =  (/<  -  l)(l/a  -  1)  is  selected  to 
satisfy  the  bound  P(ICD)  -c  a. 

The  algorithm  proceeds  as  follows: 

SE  Algorithm 


1.  Set  n  =  0. 

2.  Obtain  one  signal  from  all  area-cells  i  such  that  t'f  #  t|k)  for 
some  configurations  t®,  6k)  in  C„. 

3.  Compute  4r)  and  the  ratios  tj 4r\  Vt(r)  6  C„. 

4.  If  4/4r)  >  B,  then  remove  ttr)  from  C„. 

5.  If  |C„|  =  1,  stop  and  declare  the  remaining  configuration  in  Cn 
the  determined  configuration.  Otherwise,  increase  n  ->  n  +  1 
and  go  back  to  2. 


In  step  2  we  sample  from  any  area-cell  i  for  which  there  remain 
candidate  configurations  based  on  different  number  of  TOIs  in  AC,. 
Step  3  provides  the  constraint  to  the  types  of  problems  the  algo¬ 
rithm  can  handle.  After  the  first  round  of  looks,  the  procedure  com¬ 
putes  the  likelihood  for  all  ^  ^  +  ^  ~  ^  configurations.  Unless  M 

is  small  (e.g.  less  than  5)  the  number  of  configurations  will  make 
the  problem  computationally  infeasible.  For  example  if  the  area 
of  interest  is  a  16  x  16  grid  and  there  are  10  TOIs,  then  there  are 
over  1016  configurations.  In  practical  situations,  however,  most 
configurations  may  be  deemed  impossible,  so  that  initially  fewer 

than  ^  ^  +  A  1  j  configurations  exist. 

Let  the  stopping  time 

Nr  =  inf  |n  >  1  :  4’74r)  i  (B“’,B)} 


be  the  first  time  the  likelihood  ratio  of  Configurations  1  and  r  exits 
the  interval  (B~\B).  If  Configuration  1  is  incorrectly  eliminated  via 
the  SE  Algorithm,  then  there  must  exist  some  r  such  that 
7'rV4lr  ^  B  '•  The  probability  that  this  occurs  is 
P(Uf=24Jr74lr  <  B  ')•  Following  Eqs.  (4)-(6),  with  a  =  fl  and 
B  =  A-1  =  (/<  -  l)(l/a  -  1),  we  can  guarantee  the  error  bound 
P(4,74r;  $  B~*)  <  a/(K  -  1).  By  Bonferroni’s  inequality,  it  follows 
that  the  incorrect  determination  probability  for  the  SE  Algorithm, 
P(/CD),  will  be  less  than  a: 

P(ICD)  <  p(u<74>  <  B-'  j  <  gP(4JX’  <  B 


^Y 


a 


K  1 


=  a. 


(7) 


Hence  we  conclude  that. 


Proposition  1.  The  SE  Algorithm  is  guaranteed  to  meet  the  operator- 
defined  tolerance  a  for  the  probability  of  incorrect  determination. 


3.4.  Termination  time 


Fig.  1.  Examples  of  the  evolution  of  the  likelihood  ratio  for  the  two  configuration  In  this  section  we  show  that  the  SE  Algorithm  finishes  in  a  finite 

case.  number  of  steps  with  probability  1.  To  do  this  we  examine  the 
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behavior  of  the  likelihood  ratio  4  =  4,)/4r)  °f  two  arbitrary  and 
different  candidate  configurations  as  n  increases  and  determine 
under  what  conditions  the  likelihood  ratio  exits  a  continuation  re¬ 
gion  in  a  finite  number  of  steps. 

The  continuation  region  is  the  interval  (A,B)  and  the  stopping 
time  is  N  =  inf{n  >  \'.£n$  (A#)}.  We  will  show  that  if  each  sensor 
matrix  S,  has  no  identical  rows,  then  P[N<  oo]  =  1  for  any  finite  A 
and  B.  To  do  this  we  examine  the  log  likelihood  ratio 
logf„  =  log4,)  -  log4r>-  The  equivalent  continuation  region  for 
the  log  likelihood  ratio  is  (a,  fa)  where  a  =  logA  and  fa  =  logB  and 
the  stopping  time  can  be  written  in  terms  of  log4  via 
N  =  inf{n  >  1  :log4  f  (a, fa)}. 

We  will  express  the  log  likelihood  log4  as  a  random  walk.  That 
is  there  are  I1D  random  variables  Zj  such  that 

log  4  =  Z i  +  Z2  +  Z3  -h  .  .  .  -f  Zn. 

The  values  of  Z,  can  be  derived  by  inspecting  Eqs.  (2)  and  (3): 

ft 

zi  =  J2  (logpi(xiJ'|t',I))  -  logp^Xij-ltf1)).  (8) 

1=1 

The  Zj  are  independent  random  variables  because  each  sensor’s  sig¬ 
nals  are  independent  of  the  signals  of  other  sensors,  and  a  signal 
from  one  sensor  in  one  period  is  independent  of  a  signal  from  that 
sensor  in  another  period.  The  Zj  are  identically  distributed  because 
the  targets  are  stationary  and  we  assume  that  the  sensor  matrices  S, 
do  not  change.  Therefore,  log  4  =  £!L, 4  is  a  random  walk,  and  we 
can  appeal  to  the  theory  of  random  walks  to  prove  that 
P[N<  00]  =  1.  Because  log4  is  a  random  walk,  one  of  the  following 
four  scenarios  must  occur  with  probability  one  (see  Theorem  1.2 
in  Chapter  3.1  of  Durrett  [2]): 

1 .  log4  =  0  for  all  n 

2.  log4  ->  00 

3.  log4  — *  —00 

4.  -00  =  lim  inf  log4  <  lim  sup  log4  =  00 

For  scenarios  2,  3,  and  4,  P[N  <  00]  =  1  because  logfn  will  almost 
surely  be  growing  arbitrarily  large  (or  small),  and  thus  log4  will 
leave  the  continuation  region  (a, fa)  in  a  finite  number  of  steps  al¬ 
most  surely.  To  show  this  we  will  derive  a  contradiction.  Assume 
that  we  have  scenario  2,  but  P[N  <  00]  <  1.  We  define  the  sets  D 
and  £: 

D={co  :  lim  logfn(ffl)  =  00}, 

l  n—>  00  J 

E=  \  co  :  sup  log4(ct>)  <  b  and  inf  log4(<u)  >  a\. 

(  1@  J 

We  have  P[D]  =  1  because  we  assume  scenario  2,  and  P[£]  =  e  >  0 
because  we  assume  P[N<oo]<l.  However,  Dcf  and  therefore 
P[D]  sg  P[EC]  =  1  -  P[£]  =  1  -  e  <  1.  This  is  a  contradiction  and  so  if 
we  have  scenario  2,  then  P[N<  00]  =  1.  The  proofs  for  scenario  3 
and  4  are  nearly  identical. 

If  scenario  1  occurs  then  P[N  <  00]  =  0.  Since  P[1V  <  00]  =  1  if  sce¬ 
nario  2,  3,  or  4  occurs,  if  we  can  we  can  prove  that  scenario  1  can¬ 
not  happen,  then  by  Theorem  1.2  in  Chapter  3.1  of  Durrett  [2] 
either  scenario  2,  3,  or  4  must  occur,  and  therefore  P[JV<  00]  =  1. 
In  scenario  1,  log4  =  0  for  all  n.  This  will  happen  if  and  only  if 
Zj  =  0  for  j  =  1,2,. . .;  that  is  Zj  is  identically  equal  to  0  with  proba¬ 
bility  1.  But  this  cannot  happen  if  each  sensor  matrix  S,  does  not 
have  two  identical  rows,  because  then 

var  Zj  =  ^  var(logp,(xiJ|t<',))  -  logpf(xy|tjr)))  >  0,  (9) 

i=l;tJqWr) 


5 

since  var  (lOgp^Xylt®)  -  lOgp^Xyitf1))  >  0  whenever 
p(pi(Xij|tfVpi(xiJ|t!r)))  >0.  Eq.  (9)  implies  P(Zj  =  0)<1.  There¬ 
fore,  scenario  1  cannot  occur  in  our  model,  which  means  that  either 
scenario  2,  3,  or  4  must  occur  and  P[N<  00]  =  1. 

Since  we  have  shown  that  the  likelihood  ratio  4,)/4r)  f°r  arbi- 
trary  configurations  q  and  r  will  leave  the  continuation  region  in 
a  finite  number  of  steps  with  probability  1,  then  this  will  also  hold 
for  the  likelihood  ratio  4/4r)-  This  is  summarized  in  the  next 
proposition. 

Proposition  2.  If  each  sensor  matrix  S,  does  not  have  two  identical 
rows,  then  the  sequential  elimination  algorithm  terminates  in  finite 
time  with  probability  1. 

4.  Numerical  results 

In  Section  4.1  we  focus  on  the  sequential  eliminating  algorithm 
for  only  one  target,  and  we  compare  the  performance  of  the 
sequential  eliminating  algorithm  for  one  target  to  other  naive  algo¬ 
rithms  in  Section  4.2.  We  briefly  present  results  for  multiple  tar¬ 
gets  in  Section  4.3. 

4.1.  One  target  scenario 

We  assume  there  is  a  sensor  in  each  area-cell  that  is  available  to 
take  one  look  per  time  period.  To  construct  the  2x2  sensor  matrix 
Sj  for  these  numerical  examples  we  define  p,(0|0),  which  corre¬ 
sponds  to  the  upper  left  hand  element  of  S„  to  be  a  random  uniform 
value  between  0.7  and  1.0,  and  we  assign  a  random  uniform  value 
between  0.8  and  1.0  to  p,(l|l),  which  corresponds  to  the  lower 
right  hand  element  of  S,  (the  exact  sensor  values  used  to  generate 
the  figures  in  this  section  are  available  from  the  authors  upon  re¬ 
quest).  The  quantity  p,(  1 1 1 )  is  the  probability  the  sensor  signals 
that  one  target  is  located  in  the  area-cell  if  there  is  actually  one  tar¬ 
get  in  the  area-cell.  This  is  known  as  the  sensitivity  of  a  sensor.  The 
specificity  of  a  sensor  corresponds  to  pf(0|0),  which  is  the  probabil¬ 
ity  the  sensor  correctly  signals  that  no  targets  are  in  the  area-cell. 
The  off-diagonal  terms  of  S,,  p,( 0 1 1 )  and  Pi(l|0),  are  given  once  we 
define  the  diagonal  terms  because  the  rows  sum  to  1. 

Fig.  2  presents  the  expected  number  of  total  looks  for  the 
sequential  eliminating  procedure  to  terminate  as  a  function  of 


Fig.  2.  Expected  number  of  looks  until  the  sequential  eliminating  procedure 
terminates  as  A  increases.  There  is  one  target,  A  ranges  from  2  to  256,  a  =  0.05, 
sensor  sensitivity  ~  l/[0.8,l],  and  sensor  specificity  ~L/[0.7,1]. 
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the  number  of  area-cells  A.  We  set  the  error  tolerance  a  to  0.05.  The 
number  of  area-cells  in  the  AOI  varies  from  A  =  2  to  A  =  256.  We 
arbitrarily  (and  without  loss  of  generality)  place  the  target  in 
AC].  For  all  figures  in  this  section  we  perform  25000  replications 
for  a  given  set  of  parameter  values  to  produce  an  estimated  value; 
in  this  case  that  value  is  the  expected  number  of  looks  for  the  pro¬ 
cedure  to  return  the  determined  configuration.  The  vertical  bars 
represent  95%  of  the  distribution  of  the  number  of  looks  produced 
during  the  simulation.  Fig.  2  depicts  a  linear  increase  in  the  num¬ 
ber  of  looks  as  A  increases  for  fixed  a  =  0.05.  In  this  example 
approximately  3  looks  per  area-cell  are  required  on  average  for 
the  algorithm  to  terminate.  This  value  only  increases  by  about  1 
look  per  area-cell  if  the  total  number  of  looks  to  terminate  is  at 
the  97.5th  percentile  of  its  distribution. 

In  Fig.  3  we  examine  the  effect  of  varying  the  error  tolerance. 
We  use  the  same  sensor  parameters  as  in  Fig.  2,  fix  A  =  196,  and 
vary  the  type-I  error  probability  tolerance  a  between  0.01  and 
0.2.  As  expected,  the  number  of  expected  looks  decreases  with  an 
increase  in  error  tolerance  for  the  sequential  procedure.  The  de¬ 
crease  is  more  significant  for  small  values  of  a  because  the  thresh¬ 
old  to  eliminate  configurations  varies  inversely  with  a  (cf.,  above 
Eq.  (7)).  In  the  example  in  Fig.  3,  decreasing  the  error  tolerance 
from  0.2  to  0.01  only  requires  approximately  1.2  additional  looks 
per  area-cell  on  average. 

Finally,  the  expected  number  of  looks  until  the  sequential  elim¬ 
inating  procedure  terminates  depends  upon  the  sensor  parameters 
in  S,-.  To  determine  how  improving  the  sensors’  sensitivity  and 
specificity  impacts  the  number  of  expected  looks,  we  set  A  =  196 
and  a  =  0.05  and  vary  the  sensors’  parameters.  Therefore,  the  sen¬ 
sor  parameters  used  to  create  Fig.  4  are  different  than  the  values 
used  to  produce  Figs.  2  and  3.  We  set  the  sensor  sensitivity  to  be 
a  uniform  random  number  in  (a,a  +  0.05)  and  the  specificity  to 
be  another  uniform  random  number  in  (a, a  +  0.05)  for  a  ranging 
between  0.6  and  0.95.  As  the  probabilities  that  the  sensors  produce 
the  correct  signals  increase,  the  sequential  algorithm  terminates  in 
fewer  steps  because  there  is  less  uncertainty  with  the  sensors’  sig¬ 
nals.  Increasing  the  sensors’  sensitivity  and  specificity  has  a  signif¬ 
icant  impact  on  the  amount  of  resources  (i.e.,  looks)  required  to 
identify  the  determined  configuration.  When  the  sensors  produce 
the  correct  signal  over  95%  of  the  time,  then  less  than  2  looks  per 
area-cell  are  needed  for  the  procedure  to  finish.  However,  when 
the  signals  are  only  slightly  better  than  a  coin-flip  at  60%  the  pro¬ 


Fig.  3.  Expected  number  of  looks  until  the  sequential  eliminating  procedure 
terminates  as  a  increases.  There  is  one  target,  .A  is  196,  sensor  sensitivity  ~L/[0.8,1], 
sensor  specificity  ~  l/[0.7,l],  and  a  ranges  from  0.01  to  0.2. 


Fig.  4.  Expected  number  of  looks  until  the  sequential  eliminating  procedure 
terminates  as  the  sensitivity  and  specificity  increase.  There  is  one  target,  A  is  1 96, 
a  =  0.05,  and  the  sensitivity  and  specificity  range  from  approximately  0.6  to 
approximately  0.95. 

cedure  needs  more  than  an  order  of  magnitude  more  looks  to  fin¬ 
ish.  Fig.  4  illustrates  the  relationship. 

Fig.  4  suggests  that  one  could  perform  a  cost-benefit  compari¬ 
son  of  several  different  sensors.  “Better"  sensors  (e.g.  those  repre¬ 
sented  on  the  lower  right  of  Fig.  4)  will  terminate  in  fewer  looks 
but  they  will  presumably  have  a  higher  purchase  cost.  There  are 
two  primary  costs  associated  with  a  sensor:  the  cost  to  purchase 
and  maintain  a  sensor  and  the  cost  of  each  look  (e.g.  the  time  to 
process  a  look).  The  cost  of  one  look  may  be  quite  small,  but  if  that 
sensor  is  used  in  many  operations,  having  a  more  effective  sensor 
technology  may  produce  significant  benefits  over  time.  Another 
trade-off  analysis  that  could  be  performed  for  one  particular  sensor 
would  be  to  determine  the  optimal  location  on  the  receiver  operat¬ 
ing  characteristic  (ROC)  curve.  Often  the  operator  has  some  control 
over  the  performance  of  a  sensor;  one  can  increase  sensitivity  at 
the  expense  of  specificity  (this  differs  from  Fig.  4  where  both  sen¬ 
sitivity  and  specificity  increase).  It  would  be  valuable  to  know 
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Fig.  5.  Expected  number  of  looks  until  the  sequential  eliminating  procedure 
terminates  as  the  sensitivity  and  specificity  vary.  There  is  one  target,  A  is  196, 
a  =  0.05,  and  each  sensor  has  the  same  sensitivity  and  specificity.  The  darker  shades 
represent  a  larger  number  of  looks. 
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which  combination  of  sensitivity  and  specificity  would  produce  the 
smallest  expected  number  of  looks.  A  full  cost-benefit  analysis  of 
sensor  technology  is  outside  the  scope  of  this  paper  (and  much 
of  the  relevant  information  would  be  classified  or  difficult  to  ob¬ 
tain).  However  we  provide  one  figure  that  would  be  an  essential 
component  of  that  analysis  once  one  has  the  relevant  cost  and  sen¬ 
sor  data.  Fig.  5  presents  a  heat  map  that  plots  the  expected  number 
of  looks  as  a  function  of  the  sensitivity  and  specificity  (we  assume 
the  sensor  characteristics  are  identical  across  area  cells).  This  fig¬ 
ure  illustrates  that  there  are  primarily  eight  regions  in  the  sensitiv¬ 
ity/specificity  space  where  the  expected  number  of  looks  is  roughly 
constant  (technically  there  is  a  very  small  ninth  region  in  the  upper 
right  hand  corner  for  near  perfect  sensors  that  is  difficult  to  see  on 
the  figure).  The  expected  number  of  looks  in  the  first  region  in  the 
lower  left-hand  corner  of  Fig.  5  is  approximately  6500  and  that  val¬ 
ues  drops  by  at  least  25%  per  region  as  the  sensor  parameters  move 
to  the  northeast  until  the  value  is  less  than  500  in  the  eighth  re¬ 
gion.  Operationally,  Fig.  5  suggests  that,  in  terms  of  expected  num¬ 
ber  of  looks  until  termination,  the  marginal  values  of  the 
sensitivity  and  specificity  are  similar  when  their  values  are  alike. 
The  more  dissimilar  their  values,  the  greater  the  marginal  value 
of  the  largest  parameter  becomes.  The  structure  of  Fig.  5  would 
be  useful  in  the  evaluation  of  sensor  technologies.  For  most  situa¬ 
tions  a  small  improvement  in  sensor  performance  will  not  justify 
the  additional  cost.  However,  if  current  sensor  technology  is  on 
the  boundary  of  two  regions,  the  extra  investment  will  signifi¬ 
cantly  reduce  the  number  of  looks. 

4.2.  Comparison  with  other  target- location  algorithms 

In  this  section  we  compare  the  performance  of  the  sequential 
eliminating  procedure  with  two  other  algorithms  to  locate  targets 
in  an  AOl;  the  measure  of  performance  is  the  observed  error  rates. 
Recall  that  the  goal  of  the  sequential  eliminating  procedure  is  to 
indicate  target  location  with  an  accuracy  rate  guaranteed  to  meet 
operator-specified  error  tolerances,  within  a  reasonable  number 
of  expected  looks.  One  possible  measure  of  performance  is  the 
amount  of  slack  between  the  error  tolerance  and  the  observed  er¬ 
ror  rate.  Intuitively,  the  less  slack,  the  fewer  number  of  expected 
looks  would  be  required.  However,  if  a  method  with  the  same  ex¬ 
pected  number  of  looks  as  the  sequential  eliminating  procedure 
exhibits  a  larger  observed  error  rate,  it  is  reasonable  to  state  that 
the  sequential  eliminating  procedure  is  more  efficient  than  such 
a  method.  Alternatively,  one  could  invoke  a  method  exhibiting 
the  same  achieved  error  rate  and  compare  the  expected  number 
of  looks,  but  we  choose  the  former  scheme  for  ease  of  computation 
and  illustration. 

In  both  of  the  alternate  methods  we  consider,  we  allocate  the 
same  expected  number  of  total  looks  as  the  sequential  eliminat¬ 
ing  procedure.  For  given  values  of  S,  and  a,  let  xA  be  the  total 
average  number  of  looks  for  the  sequential  eliminating  procedure 
to  complete  with  an  AOI  consisting  of  A  area-cells.  We  compute 
the  average  number  of  looks  numerically  in  Figs.  2-4.  In  the  naive 
procedure,  we  allocate  the  same  number  of  looks  to  each  sensor 
so  that  each  area-cell  receives  ta/A  looks.  In  a  slightly  more 
sophisticated  procedure,  we  allocate  the  looks  to  area-cells  based 
on  how  effective  their  sensors  are.  Looks  are  allocated  in  propor¬ 
tion  to  the  inverse  of  (sensitivity  -  (1 -specificity)).  If  this  quantity 
is  large  (that  is  the  sensor  has  high  sensitivity  and  specificity)  the 
sensor  is  efficient  and  needs  less  looks  to  provide  useful  informa¬ 
tion  to  the  operator.  If  the  difference  between  sensitivity  and  the 
Type-II  error  of  the  sensor  is  small  then  it  is  difficult  to  interpret 
one  signal  with  great  certainty,  and  therefore  more  signals  from 
those  sensors  are  collected.  If  AC,  should  receive  a  fraction/,  looks 
according  to  this  procedure,  then  fxA  looks  will  be  allocated  to 
that  area-cell. 


7 

In  both  the  procedure  that  equally  allocates  looks  to  every  area¬ 
cell  and  the  procedure  that  allocates  looks  in  proportion  to  the  in¬ 
verse  of  sensor  effectiveness,  the  calculated  looks-per-cell  will 
likely  not  be  integral.  We  round  the  calculated  value  up  to  the 
nearest  integer,  and  therefore  these  two  alternate  procedure  will 
have  more  total  looks  than  the  sequential  eliminating  procedure. 
After  the  deterministic  allocation  of  looks  to  the  area-cells,  the  con¬ 
figuration  with  the  largest  likelihood  is  declared  the  determined 
configuration  for  each  of  the  algorithms.  Unlike  the  sequential 
eliminating  procedure,  neither  configurations  nor  area-cells  are 
eliminated  during  these  two  alternate  procedures. 

In  Fig.  6  we  compare  the  observed  error  rates  of  the  sequential 
eliminating  procedure  with  that  of  the  alternate  models.  We  set 
a  =  0.05  and  vary  the  number  of  area-cells  from  A  =  2  to  A  =  256. 
As  before,  the  target  is  located  in  AC]  and  the  sensor  parameters 
are  the  same  ones  used  produce  Fig.  2.  Furthermore,  for  the 
sequential  eliminating  procedure  we  use  the  same  data  that  pro¬ 
duced  Fig.  2.  We  do  not  include  vertical  bars  in  Figs.  6-8  because 
we  are  only  computing  one  realization  of  the  error  rate  (as  opposed 
to  25000  realizations  of  the  number  of  looks  in  Figs.  2-4).  As  con¬ 
structed,  the  observed  error  rate  of  the  sequential  eliminating  pro¬ 
cedure  is  less  than  the  error  bound  a  (significantly  so  in  this  case, 
due  to  the  slack  introduced  by  Bonferroni’s  inequality,  cf.  Eq.  (7)). 
For  the  same  average  number  of  looks  at  each  value  of  A,  the  ob¬ 
served  error  rates  for  the  two  alternate  methods  are  considerably 
larger  for  all  values  of  A  (indeed,  the  alternate  error  rates  exceed 
the  tolerance  cl  for  most  values  of  A).  In  many  situations  the  algo¬ 
rithm  that  allocates  looks  inversely  to  sensor  effectiveness  pro¬ 
duces  more  errors  than  the  algorithm  that  allocates  an  equal 
number  of  looks  to  each  area-cell.  By  potentially  allocating  too 
many  looks  to  too  few  area-cells  the  former  algorithm  does  not  col¬ 
lect  enough  information  from  the  remaining  area-cells  to  accu¬ 
rately  produce  the  correct  configuration. 

We  next  examine  the  effect  of  varying  a  with  A  =  196.  Fig.  7 
shows  that  an  increase  in  a.  appears  to  cause  a  linear  increase  in 
the  observed  miss  rate.  The  observed  miss  rate  for  a  =  0.2  is  still 
much  less  than  0.05.  The  bound  derived  in  (7)  is  not  tight,  and  as 
Fig.  7  illustrates  there  is  a  large  amount  of  slack  between  the  de¬ 
sired  error  rate  and  the  observed  error  rate.  This  suggests  there 
is  potential  to  improve  the  sequential  eliminating  procedure  by 
terminating  after  a  fewer  number  of  looks.  The  difference  between 
the  observed  error  rate  for  the  sequential  eliminating  procedure 


Fig.  6.  Observed  error  rates  of  the  target-location  algorithms  as  A  increases.  There  is 
one  target,  A  ranges  from  2  to  256,  a  =  0.05,  sensor  sensitivity  ~ti[0.8, 1  ],  and  sensor 
specificity  ~tl[0.7, 1  ]. 
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Fig.  7.  Observed  error  rates  of  the  target-location  algorithms  as  a  increases.  There  is 
one  target,  A  is  196,  sensor  sensitivity  ~l/[0.8, 1  ],  sensor  specificity  ~U[0.7, 1  ],  and  cc 
ranges  from  0.01  to  0.2. 


depending  upon  the  circumstances  (e.g.  the  sensor  technology, 
environment,  nature  of  the  targets,  and  ability  of  the  operators), 
the  sensors’  signals  will  be  unreliable  and  the  sensitivity  and  spec¬ 
ificity  will  be  much  less  than  0.9.  These  are  the  situations  where 
the  sequential  eliminating  procedure  can  produce  enormous  effi¬ 
ciency  gains. 

4.3.  Multiple  targets 

The  results  for  multiple  targets  are  similar  to  the  case  of  one  tar¬ 
get.  Fig.  9  presents  how  the  expected  number  of  looks  for  the 
sequential  eliminating  procedure  to  terminate  varies  as  the  num¬ 
ber  of  targets,  M,  increases.  We  set  a  =  0.05,  A  =  10,  and  vary  M 
from  1  to  10.  We  define  the  GTC  to  be  one  target  in  the  first  M 
area-cells.  There  are  many  ways  the  sensors’  capabilities  can 
change  as  the  number  of  potential  targets  increases.  We  examine 
two  cases.  In  one  case  a  sensor’s  ability  to  signal  the  correct  num¬ 
ber  of  targets  in  the  area-cell  is  a  uniform  random  variable  be¬ 
tween  0.5  and  1  (these  are  the  diagonal  values  in  S,).  This  means 
that,  regardless  of  the  number  of  targets,  the  sensors  produce  a 
correct  signal  at  least  50%  of  time.  The  remaining  sensor  parame¬ 
ters  are  uniformly  generated  to  ensure  that  the  rows  are  pmfs 
(i.e.,  the  rows  sum  to  1).  We  also  examine  a  case  where  the  prob¬ 
ability  a  sensor  produces  a  correct  signal  decreases  as  the  number 
of  targets  increases.  For  simplicity,  in  this  case  we  assume  that  the 
off-diagonal  elements  of  S,  are  identical  and  that  the  diagonal  ele¬ 
ments  are  a  multiple  of  the  off-diagonal  terms.  We  assume  the 
diagonal  element  of  each  row  of  Si  is  a  factor  of  4  larger  than  an 
off-diagonal  element  because  this  corresponds  to  a  sensitivity 
and  specificity  of  0.8  when  there  is  one  target.  We  further  assume 
the  parameters  of  every  sensor  are  the  same,  and  therefore  the  off- 
diagonal  elements  of  S,  equal  1/(M  +  4)  and  the  diagonal  elements 
of  St  equal  4/(M  +  4). 

Adding  additional  targets  rapidly  increases  the  number  of  feasi¬ 
ble  configurations.  With  one  target  there  are  A  configurations  (i.e., 
the  target  can  be  in  one  of  the  area-cells).  However,  with  5  targets 
in  10  area-cells  there  are  over  2000  configurations.  Not  only  are 
there  more  configurations  to  consider,  but  the  threshold  to  elimi¬ 
nate  configurations  increases  with  the  number  of  feasible  configu¬ 
rations  (cf.,  above  Eq.  (7)).  It  might  be  expected  that,  as  with  the 
number  of  configurations,  the  number  of  looks  would  increase 
exponentially  as  we  increase  M.  However,  for  the  two  cases  we 


Fig.  8.  Observed  error  rates  of  the  target-location  algorithms  as  the  sensitivity  and 
specificity  increase.  There  is  one  target,  A  is  196,  a  =  0.05,  and  the  sensitivity  and 
specificity  range  from  approximately  0.6  to  approximately  0.95 


and  the  two  alternate  procedures  is  significant:  the  error  rates  for 
the  latter  two  start  at  over  15%  and  increase  to  over  25%. 

Finally  we  present  the  companion  to  Fig.  4  in  Fig.  8  for  ot  =  0.05 
with  A  =  196.  As  the  sensors  improve  and  they  more  accurately  sig¬ 
nal  the  true  nature  of  the  area-cells,  the  error  rates  of  all  the  pro¬ 
cedures  generally  decrease.  Although  the  observed  error  rates  of 
the  two  alternate  procedures  are  still  greater  than  a  =  0.05,  except 
when  the  sensitivity  and  specificity  are  greater  than  0.95.  It  is  not 
surprising  that  all  three  algorithms  have  relatively  low  error  rates 
when  the  sensitivity  and  specificity  are  both  large  because  often 
one  or  two  looks  per  area-cell  are  sufficient  to  produce  a  deter¬ 
mined  configuration.  However,  when  sensors  are  not  effective 
and  the  difference  between  the  probability  of  a  true  positive  and 
a  false  positive  gets  closer  to  zero,  the  dominance  of  the  sequential 
eliminating  procedure  over  the  alternate  methods  is  staggering. 
The  alternate  methods  produce  nearly  as  many  incorrect  configu¬ 
rations  as  true  configurations,  whereas  the  sequential  eliminating 
procedure  is  still  comfortably  below  the  a  =  0.05  error  rate.  Often, 
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Fig.  9.  Expected  number  of  looks  until  the  sequential  eliminating  procedure 
terminates  as  M  increases  for  two  different  categories  of  the  sensor  matrices  S,. 
A  =  10,  a  =  0.05  and  M  ranges  from  1  to  10. 
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examine  in  Fig.  9  this  does  not  occur.  In  fact  when  the  sensor  cor¬ 
rectly  identifies  the  target  over  50%  of  the  time,  the  average  num¬ 
ber  of  looks  is  relatively  constant  as  M  increases.  This  is  because 
the  probability  of  returning  a  specific  incorrect  signal  is  much  less 
than  when  there  is  only  one  target.  Therefore,  the  configurations 
that  do  not  match  most  of  the  sensor  signals  will  quickly  be  elim¬ 
inated.  When  the  probability  that  a  sensor  produces  the  correct 
signal  decreases  as  M  increases,  the  number  of  looks  appears  to  in¬ 
crease  linearly.  In  this  case,  additional  targets  cause  the  sensors  to 
produce  more  incorrect  signals,  and  therefore  to  overcome  this 
uncertainty  the  sensors  must  take  additional  looks.  In  the  example 
in  Fig.  9,  every  additional  target  requires  approximately  4  more 
looks  per  area-cell  to  find  the  determined  configuration. 

When  multiple  targets  are  located  in  the  AOI,  the  observed  error 
rates  are  similar  to  the  results  presented  in  Figs.  6-8.  The  sequen¬ 
tial  eliminating  procedure  produces  errors  at  a  rate  much  less  than 
a.  and  it  dominates  the  two  alternate  procedures.  However,  the  er¬ 
ror  rates  of  the  two  alternate  procedures  are  smaller  then  when 
there  is  only  one  target.  Because  of  the  large  number  of  configura¬ 
tions,  the  threshold  to  eliminate  configurations  is  greater  than  it 
needs  to  be  to  achieve  the  desired  error  rate,  which  causes  consid¬ 
erable  slack  between  the  observed  and  desired  error  rates. 

5.  Concluding  remarks 

The  sequential  eliminating  procedure  collects  signals  from  sen¬ 
sors  and  determines  how  targets  are  located  in  area-cells  in  an  area 
of  interest.  This  procedure  provides  efficient  results  guaranteed  to 
meet  desired  error  rate  bounds.  The  key  assumptions  are  that  tar¬ 
gets  are  nonreactive  and  static,  or  that  the  targets  remain  in  the 
same  area-cell  during  the  time-scale  of  interest. 

We  show  that  two  naive,  non-eliminating  methods  demon¬ 
strate  consistently  higher  error  rates  for  the  same  total  number 
of  looks  for  all  appropriate  values  of  number  of  area-cells,  error  tol¬ 
erances,  and  sensor  parameters.  There  is  also  no  guarantee  that  the 
naive  procedures’  results  meet  the  operator-prescribed  error  toler¬ 
ance.  In  fact,  for  most  of  the  examples  we  analyze,  their  error  rates 
are  higher  than  the  tolerance.  It  is  therefore  reasonable  to  conclude 
that  the  sequential  eliminating  procedure  is  more  efficient  than  the 
naive  allocation  methods.  The  sequential  eliminating  procedure  is 
especially  superior  to  the  other  two  methods  when  the  sensors 
have  lower  sensitivity  and  specificity  (cf.,  Fig.  8).  The  number  of 
looks  scales  linearly  with  the  the  number  of  area-cells  and  targets 
for  the  examples  in  Section  4.  This  suggests  the  sequential  elimi¬ 
nating  procedure  can  be  utilized  in  scenarios  involving  a  large  re¬ 
gion  with  many  targets. 

The  sequential  eliminating  procedure  can  be  useful  even  if  it 
cannot  run  until  completion.  This  may  occur  if  there  are  a  limited 
number  of  looks  available  due  to  resource  or  time  constraints.  The 
sequential  eliminating  procedure  can  run  until  the  look  limit  is 
reached,  and  then  with  probability  1-a  the  targets  are  located 
according  to  one  of  the  remaining  candidate  configurations.  Fur¬ 
ther  investigation  by  different  search  assets  (e.g.  interceptor 


teams)  may  be  required  to  determine  the  location  of  the  targets. 
The  sequential  eliminating  procedure  reduces  the  possibilities  to 
the  remaining  candidate  configurations.  If  one  determined  config¬ 
uration  must  be  returned  at  the  end  of  the  procedure,  then  the  con¬ 
figuration  with  the  largest  likelihood  can  be  chosen. 

Finally,  the  analysis  in  this  paper  presents  the  worst  case  per¬ 
formance  in  some  sense.  The  sequential  eliminating  procedure  ter¬ 
minates  only  when  one  configuration  remains,  and  we  consider  an 
incorrect  determination  to  occur  if  the  determined  configuration  is 
not  exactly  the  ground  truth  configuration.  The  procedure  could 
return  a  set  of  candidate  configurations  rather  than  one  deter¬ 
mined  configuration,  and  for  many  applications  several  acceptable 
configurations  may  exist.  For  example  the  goal  may  be  to  deter¬ 
mine  which  area-cells  contain  any  targets;  the  exact  distribution 
of  the  targets  in  the  area-cells  may  not  be  important.  Our  analysis 
provides  an  upper  bound  on  the  error  rate  and  number  of  looks  if 
there  are  multiple  acceptable  configurations. 
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